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Ambient Light Derived from Video Content by Mapping 
Transformations Through Unrendered Color Space 

This invention relates to production and setting of ambient lighting effects using 
5 multiple light sources, and typically based on, or associated with, video content, such as 
from a video display. More particularly, it relates to a method to drive or set multiple 
ambient light sources by extracting selected color information from video in real time, and 
performing color mapping transformations from the video environment to that which 
allows driving a plurality of ambient light sources. 

10 

Engineers have long sought to broaden the sensory experience obtained 
consuming video content, such as by enlarging viewing screens and projection areas, 
modulating sound for realistic 3 -dimensional effects, and enhancing video images, 
including broader video color gamuts, resolution, and picture aspect ratios, such as with 

15 high definition (HD)'digital TV television and video systems. Moreover, film, TV, and 

video producers also try to influence the experience of the viewer using visual and auditory 
means, such as by clever use of color, scene cuts, viewing angles, peripheral scenery, and 
computer-assisted graphical representations. This would include theatrical stage lighting 
as well. Lighting effects, for example, are usually scripted - synchronized with video or 

20 play scenes - and reproduced with the aid of a machine or computer programmed with the 
appropriate scene scripts encoded with the desired schemes. 

In the prior art digital domain, automatic adaptation of lighting to fast changes in 
a scene, including unplanned or unscripted scenes, has not been easy to orchestrate in large 
part because of the overhead of large high bandwidth bit streams required using present 

25 systems. 

Philips (Netherlands) and other companies have disclosed means for changing 
ambient or peripheral lighting to enhance video content for typical home or business 
applications, using separate light sources far from the video display, and for many 
applications, some sort of advance scripting or encoding of the desired lighting effects. - 
30 Ambient lighting added to a video display or television has been shown to reduce viewer 
fatigue and improve realism and depth of experience. 

Sensory experiences are naturally a function of aspects of human vision, which 
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uses an enormously complex sensory and neural apparatus to produce sensations of color 
and light effects. Humans can distinguish perhaps 10 million distinct colors. In the human 
eye, for color-receiving or photopic vision, there are three sets of approximately 2 million 
sensory bodies called cones which have absorption distributions which peak at 445, 535, 
5 and 565 nm light wavelengths, with a great deal of overlap. These three cone types form 
what is called a tristimulus system and are called B (blue), G (green), and R (red) for 
historical reasons; the peaks do not necessarily correspond with those of any primaiy 
colors used in a display, e.g., commonly used RGB phosphors. There is also interaction for 
scotopic, or so-called night vision bodies called rods. The human eye typically has 120 

10 million rods, which influence video experiences, especially for low light conditions such as 
found in a home theatre. 

Color video is founded upon the principles of hxmian vision, and well known 
trichromatic and opponent channel theories of human vision have been incorporated into 
our understanding of how to influence the eye to see desired colors and effects which have 

^ 1 5 high fidelity to an original or intended image. In most color models and spaces, three 
dimensions or coordinates are used to describe human visual experience. 

Color video relies absolutely on metamerism, which allows production of color 
perception using a small number of reference stimuli, rather than actual light of the desired 
color and character. In this way, a whole gamut of colors is reproduced in the human mind . 

20 using a limited number of reference stimuli, such as well known RGB (red, green, blue) 
tristimulus systems used in video reproduction worldwide. It is well known, for example, 
that nearly all video displays show yellow scene light by producing approximately equal 
amounts of red and green light in each pixel or picture element. The pixels are small in 
relation to the solid angle they subtend, and the eye is fooled into perceiving yellow; it 

25 does not perceive the green or red that is actually being broadcast. 

There exist many color models and ways of specifying colors, including well 
known CIE (Commission Internationale de i'Eclairage) color coordinate systems in use to 
describe and specify color for video reproduction. Any number of color models can be 
employed using the instant invention, including application to unrendered opponent color 

30 spaces, such as the CIE L*U*V* (CIELUV) or CIE L*a*b* (CIELAB) systems. The CIE 
established in 1931 a foundation for all color management and reproduction, and the result 
is a chromaticity diagram which uses three coordinates, x, y, and z. A plot of this three 
dimensional system at maximum luminosity is universally used to describe color in terms 
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of X and y, and this plot, called the 193 1 x,y chromaticity diagram, is believed to be able to 
describe all perceived color in humans. This is in contrast to color reproduction, where 
metamerism is used to fool the eye and brain. Many color models or spaces are in use today 
for reproducing color by using three primary colors or phosphors, among them Adobe RGB, 
5 NTSC RGB, etc. 

It is important to note, however, that the range of all possible colors exhibited by 
video systems using these tristimulus systems is limited. The NTSC (National Television 
Standards Comumittee) RGB system has a relatively wide range of colors available, but this 
system can only reproduce half of ail colors perceivable by humans. Many blues and 

10 violets, blue-greens, and oranges/reds are not rendered adequately using the available scope 
of traditional video systems. 

Furthermore, the human visual system is endowed with qualities of compensation 
and discernment whose understanding is necessary to design any video system. Color in 
humans can occur in several modes of appearance, among them, object mode and illuminant 

15 mode. 

In object mode, the light stimulus is perceived as light reflected from an object 
illuminated by a light source. In illuminant mode, the light stimulus is seen as a source of 
light. Illuminant mode includes stimuli in a complex field that are much brighter than other 
stimuli. It does hot include stimuli known to be light sources, such as video displays, whose 

20 brightness or luminance is at or below the overall brightness of the scene or field of view so 
that the stimuli appear to be in object mode. 

Remarkably, there are many colors which appear only in object mode, among 
them, brown, olive, maroon, grey, and beige flesh tone. There is no such thing, for 
example, as a brown illuminant source of light, such as a browrhcolored traffic light. 

25 For this reason, ambient lighting supplements to video systems which attempt to 

add object colors cannot do so using direct sources of bright light No combination of 
bright red and green sources of light at close range can reproduce brown or maroon, and this 
limits choices considerably. Only spectral colors of the rainbow, in varying intensities and 
saturation, can be reproduced by direct observation of bright sources of light. This 

30 underscores the need for fine control over ambient lighting systems, such as to provide low 
intensity luminance output from light sources with particular attention to hue management. 
This fine control is not presently addressed in a way that permits fast-changing and subtle 
ambient lighting under present data architectures. 
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Video reproduction can take many forms. Spectral color reproduction allows 
exact reproduction of the spectral power distributions of the original stimuli, but this is not 
realizable in any video reproduction that uses three primaries. Exact color reproduction can 
replicate human visual tristimulus values, creating a metameric match to the original, but 
5 overall viewing conditions for the picture and the original scene must be similar to obtain a 
similar appearance. Overall conditions for the picture and original scene include the 
angular subtense of the picture, the luminance and chromaticity of the surround, and glare. 
One reason that exact color reproduction often cannot be achieved is because of limitations 
on the maximum luminance that can be produced on a color monitor. 

1 0 Colorimetric color reproduction provides a useful alternative where tristimulus 

values are proportional to those in the original scene. Chromaticity coordinates are 
reproduced exactly, but with proportionally reduced luminances. Colorimetric color 
reproduction is a good reference standard for video systems, assuming that the origmal and 
the reproduced reference whites have the same chromaticity, the viewing conditions are the 

1 5 same, and the system has an overall gamma of unity. Equivalent color reproduction, where 
chromaticity and luminances match the original scene cannot be achieved because of the 
limited luminance generated in video displays. 

Most video reproduction in practice attempts to achieve corresponding color 
reproduction, where colors reproduced have the same appearance that colors in the original 

20 would have had if they had been illuminated to produce the same average luminance level 
and the same reference white chromaticity as that of the reproduction. Many, however, 
argue that the ultimate aim for display systems is in practice preferred color reproduction, 
where preferences of the viewer influence color fidelity. For example, suntanned skin color 
is preferred to average real skin color, and sky is preferred bluer and foliage greener than 

25 they really are. Even if corresponding color reproduction is accepted as a design standard, 
some colors are more important than others, such as flesh tones, the subject of special 
treatment in many reproduction systems such as the NTSC video standard. 

In reproducing scene light, chromatic adaptation to achieve white balance is 
important. With properly adjusted cameras and displays, whites and neutral grays are 

30 typically reproduced with the chromaticity of CIE standard daylight illuminant D6S. By 

always reproducing a white surface with the same chromaticity, the system is mimicking the 
human visual system, which inherently adapts perceptions so that white surfaces always 
appear the same, whatever the chromaticity of the illuminant, so that a white piece of paper 
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will appear white, whether it is found in a bright sunlight day at the beach, or a 
incandescent-lit indoor scene. In color reproduction, white balance adjustment usually is 
made by gain controls on the R, G, and B channels. 

The light output of a typical color receiver is typically not linear, but rather 
5 follows a power-law relationship to applied video voltages. The light output is proportional 
to the video-driving voltage raised to the power gamma, where gamma is typically 2.5 for a 
color CRT (cathode ray tube), and 1 .8 for other types of light sources. Compensation for 
this factor is made via three primary gamma correctors in camera video processing 
amplifiers, so that the primary video signals that are encoded, transmitted and decoded are 

10 in fact not R, G, and B, but R^^^, G^^^, and B^^^. Colorimetric color reproduction requires that 
the overall gamma for video reproduction - including camera, display, and any gamma- 
adjusting electronics be unity, but when corresponding color reproduction is attempted, the 
luminance of the surround take precedence. For example, a dim surround requires a gamma 
of about 1 .2, and a dark surround requires a gamma of about 1 .5 for optimum color 

1 5 reproduction. Gamma is an important implementation issue for RGB color spaces. . 

Most color reproduction encoding uses standard RGB color spaces, such as 
sRGB, ROMM RGB, Adobe RGB 98, Apple RGB, and video RGB spaces such as that used 
in the NTSC standard. Typically, an image is captured into a sensor or source device space, 
which is device and image specific. It may be transformed into an luirendered image space, 

20 which is a standard color space describing the original's colorimetry (see Definitions 
section). 

However, video images are nearly always directly transformed fi*om a source 
device space into a rendered image space (see Definitions section), which describes the 
color space of some real or virtual output device such as a video display. Most existing 

25 standard RGB color spaces are rendered image spaces. For example, source and output 

spaces created by cameras and scanners are not CIE-based color spaces, but spectral spaces 
defined by spectral sensitivities and other characteristics of the camera or scanner: 

Rendered image spaces are device-specific color spaces based on the colorimetry 
of real or virtual device characteristics. Images can be transformed into rendered spaces 

30 fi-om either rendered or unrendered image spaces. The complexity of these transforms 
varies, and can include complicated image dependent algorithms. The transforms can be 
non-reversible, with some information of the original scene encoding discarded or 
compressed to fit the dynamic range and gamut of a specific device. 
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There is currently only one unrendered RGB color space that is in the process of 
becoming a standard, ISO RGB defined in ISO 17321, most often used for color 
characterization of digital still cameras. In most applications today, images are converted 
into a rendered color space for either archiving and data transfer, including video signals. 
5 Converting from one rendered image or color space to another can cause cause severe 
image artifacts. The more mismatched the gamuts and white points are between two 
devices, the stronger the negative effects. 

One problem in prior art ambient light display systems is that no specific method 
is given to provide for synchronous real time operation to transform rendered tristimulus 

1 0 values from video to that of ambient light sources to give proper colorimetry and 

appearance. For example, output from LED ambient light sources is often garish, with 
limited or skewed color gamuts, and hue and chroma are difficult to assess and reproduce. 
For example, US Patent 6,61 1,297 to Akashi et al. deals with realism in ambient lighting, 
but no specific method is given to insure correct and pleasing chromaticity, and the teaching. 

15 of Akashi *297 does not allow for anedyzing video in real time, but rather needs a script or 
the equivalent. 

In addition, setting of ambient light sources using gamma corrected color spaces 
from video content often result in garish, bright colors. A more serious problem in the prior 
art is the large amount of transmitted information that is needed to drive ambient light 
20 sources as a function of real time video content, and to suit a desired fast-changing ambient 
light environment where good color matching is desired. 

It is therefore advantageous to expand the possible gamut of colors produced by 
ambient lighting in conjunction with a typical tristimulus video display system. It is also 
desired to exploit characteristics of the human eye, such as changes in relative luminosity of 
25 different colors as a function of light levels, by modulating or changing color and light 
character delivered to the video user using an ambient lighting system that uses to good 
advantage compensating effects, sensitivities, and other peculiarities of human vision. 

It is also advantageous to create a quality ambient atmosphere free from the 
effects of gamma-induced distortion. It is further desired to be able to provide a method for 
30 providing emulative ambient lighting drawn from selected video regions using an 
economical data stream that encodes average or characterized color values. 

Information about video and television engineering, compression technologies, 
data transfer and encoding, human vision, color science and perception, color spaces. 
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colorimetry and image rendering, including video reproduction, can be found in the 
following references which are hereby incorporated into this disclosure in their entirety: 
ref[l] Color Perception, Alan R. Robertson, Physics Today, December 1992, Vol 45, No 12, 
pp. 24-29; ref[2] The Physics and Chemistry of Color, 2ed, Kurt Nassau, John Wiley & 
5 Sons, Inc., New York © 2001 ; refI3] Principles of Color Technology, 3ed, Roy S. Bems, 
John Wiley & Sons, Inc., New York, © 2000; ref[4] Standard Handbook of Video and . 
Television Engineering, 4ed, Jerry Whitaker and K. Blair Benson, McGraw-Hill, New York 
© 2003. 

The invention relates to a method for extracting and processing video content 
1 0 encoded in a rendered color space to be emulated by an ambient light source, comprising [1] 
Extracting color information from a video signal that encodes at least some of the video 
content in the rendered color space; [2] Transforming the color information to an 
unrendered color space; [3] Transforming the color information from the unrendered color 
space to a second rendered color space so formed as to allow driving the ambient light 
15 source. 

Step [1] can additionally comprise decoding the video signal into a set of frames; 
extracting an average color from the color information, including at least one extraction of 
the color information from an extraction region; using the extraction of the color 
information to broadcast ambient light from the ambient light source adjacent the extraction 
20 region. In addition, one can perform a gamma correction to the second rendered color space 
fed to the ambient light units. 

Steps [2] and [3] can additionally comprise matrix transformations of primaries 
of the rendered color space and second rendered color space to the unrendered color space 
using first and second tristimulus primary matrices;and deriving a transformation of the 
25 color information into the second rendered color space by matrix multiplication of the 
primaries of the rendered color space, the first tristimulus matrix, and the inverse of the 
second tristimulus matrix. 

The unrendered color space can be one of CIE XYZ; ISO RGB defined in ISO 
Standard 17321; Photo YCC; and CIE LAB, and steps [1], [2], and [3] can be substantially 
30 synchronous with the video signal, with ambient light broadcast from or around the video 
display using the color information in the second rendered color space. 

Another method is disclosed for extracting and processing border region video 
content from a rendered color space to be emulated by an ambient light source, comprising: 
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[1] Extracting color information from a video signal that encodes at least some of 
the video content in the rendered color space, after decoding the video signal into individual 
frames; 

[2] Extracting an average color from the color information from an extraction region 
5 in each of the individual frames; [3] Transforming the average color to an unrendered color 
space; [4] Transforming the average color from the unrendered color space to a second 
rendered color space so formed as to allow driving the ambient light source; and [5] using 
the average color to broadcast ambient light from the ambient light source adjacent the 
extraction region. Steps [1], [2], [3], [4], and [5] can be substantially synchronous with the 
10 video signal. 

. In addition, a method is disclosed for extracting and processing border region 
video content from a rendered color space to be emulated by an ambient light source, using 
a colorimetric estimate, comprising: [1] Extracting color information from a video signal 
that encodes at least some of the video content in the rendered color space, after decoding 

15 the video signal into individual frames; [2] Extracting a colorimetric estimate from the color 
information from an extraction region in each of the individual frames; [3] Transforming 
the colorimetric estimate to an unrendered color space; [4] Transforming the colorimetric 
estimate from the unrendered color space to a second rendered color space so formed as to 
allow driving the ambient light source; and [5] using the colorimetric estimate to broadcast 

20 ambient light (L4) from the ambient light source adjacent the extraction region. 

FIG. 1 shows a simple front surface view of a video display showing color 
information extraction regions and associated broadcasting of ambient light from six 
ambient light sources according to the invention; 

FIG. 2 shows a downward view — part schematic and part cross-sectional — of a 

25 room in which ambient light from multiple ambient light sources is produced using the 
invention; 

FIG. 3 shows a system according to the invention to extract color information 
and effect color space transformations to allow driving an ambient light source; 

FIG. 4 shows an equation for calculating average color information from a video 
30 extraction region; 

FIG. 5 shows a prior art matrix equation to transform rendered primaries RGB 
into unrendered color space XYZ; 

FIGS. 6 and 7 show matrix equations for mapping video and ambient lighting 
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rendered color spaces, respectively, into unrendered color space; 

FIG. 8 shows a solution using known matrix inversion to derive ambient light 
tristimulus values R'G'B' from unrendered color space XYZ; 

FIGS. 9-11 show prior art derivation of tristimulus primary matrix M using a 
5 white point method; 

FIG. 12 shows a system similar to that shown in FIG. 3, additionally comprising 
a gamma correction step for ambient broadcast; 

FIG. 13 shows a schematic for a general transformational process used in the 

invention; 

10 FIG. 14 shows an extraction step as shown in FIGS. 3 and 12, employing a frame 

decoder and performing an output calculation for driving an ambient light source; 

FIG. 15 shows process steps for color information extraction and processing for 
the invention; 

FIG. 16 shows process steps for acquiring transformation matrix coefficients for 
15 an ambient light source used by the invention; 

FIG. 17 shows process steps for estimated video extraction and ambient light 
reproduction using the invention. 

The following definitions shall be used throughout: 

- Ambient light source — shall, in the appended claims, include any lighting 
20 production circuits or drivers needed to decode a light script code for use thereby. 

— Ambient space — shall connote any and all material bodies or air or space external 
to a video display unit. 

- Average color - shall, in the appended claims, include average characterizations 
other than numerical averages, and shall include functional or operator-defined 

25 characterizations of video content, including offsets of chromaticities and luminances. 

~ Chrominance - shall, in the context of driving an ambient light source, denote a 
mechanical, numerical, or physical way of specifying the color character of light produced, 
and shall not imply a particular methodology, such as that used in NTSC or PAL television 
broadcasting. 

30 - Color information - shall include either or both of chrominance and luminance, 

or functionally equivalent quantities. 

— Computer - shall include not only all processors, such as CPU's (Central 
Processing Units) that employ known architectures, but also any intelligent device that can 
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allow coding, decoding, reading, processing, execution of setting codes or change codes, 
such as digital optical devices, or analog electrical circuits that perform the same functions. 

- Controlled operating parameter - shall denote a parameter encoded as a 
representation of a physical quantity or physical variable, such as a luminance, a 

5 chrominance, or a light character index such as a delivery angle or a goniophotometric 
index. 

— Goniochromatic - shall refer to the quality of giving different color or 
chromaticity as a function of viewing angle or angle of observation, such as produced by 
iridescence. 

10 - Goniophotometric - shall refer to the quality of giving different light intensity,, 

transmission and/or color as a function of viewing angle or angle of observation, such as 
found in pearlescent, sparkling or retroreflective phenomena. 

- Interpolate - shall include linear or mathematical interpolation between two sets 
of values, as well as functional prescriptions for setting values between two known sets of 

15 values.... 

— Light character — shall mean, in the broad sense, any specification of the nature 
of light-such as produced by an ambient light source, including all descriptors other than 
luminance and chrominance, such as the degree of light transmission or reflection; or any 
specification of goniophotometric qualities, including the degree to which colors, sparkles, 

20 or other known phenomena are produced as a function of viewing angles when observing an 
ambient light source; a light output direction, including directionality as afforded by 
specifying a Poynting or other propagation vector; or specification of angular distribution of 
light, such as solid angles or solid angle distribution functions. It can also include a 
coordinate or coordinates to specify locations on an ambient light source, such as element 

25 pixels or lamp locations. 

- Luminance - shall denote any parameter or measure of brightness, intensity, or 
equivalent measure, and shall not imply a particular method of light generation or 
measurement, or psycho -biological interpretation. 

— Rendered color space — shall denote an image or color space captured from a 
30 sensor, or specific to a source or display device, which is device and image-specific. Most 

RGB color spaces are rendered image spaces, including the video spaces used to drive video 
display D. In the appended claims, both the color spaces specific to the video display and 
the ambient light source 88 are rendered color spaces. 
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— Transforming color information to an unrendered color space — in the 

appended claims shall comprise either direct transformation to the unrendered color space, 
or use or benefit derived from using inversion of a tristimulus primary matrix obtained by 
transforming to the unrendered color space (e.g., (Ma/^ as shown in FIG. 8). 
5 — Unrendered color space — shall denote a standard or non-device-specific color 

space, such as those describing original image colorimetry usmg standard CIE XYZ; ISO 
RGB, such as defined in ISO 17321 standards; Photo YCC; and the CIE LAB color space. 

— Video - shall denote any visual or light producing device, whether an active 
device requiring energy for light production, or any transmissive medium which conveys 

1 0 image information, such as a window in an office building, or an optical guide where image 
information is derived remotely. 

— Video signal — shall denote the signal or information delivered for controlling a 
video display unit, including any audio portion thereof. It is therefore contemplated that 
video content analysis includes possible audio content analysis for the audio portion. 

15 Generally, a video signal can comprise any type of signal, such as radio frequency signals 
using any number of known modulation techniques; electrical signals, including analog and 
quanitized analog waveforms; digital (electrical) signals, such as those using pulse-width 
modulation, pulse-number modulation, pulse-position modulation, PCM (pulse code 
modulation) and pulse amplitude modulation; or other signals such as acoustic signals, 

20 audio signals, and optical signals, all of which can use digital techniques. Data that is 
merely sequentially placed among or with other information, such as in computer-based 
applications, can be used as well. 

Ambient light derived from video content according to the invention is formed to 
allow, if desired, a high degree of fidelity to the chromaticity of original video scene light, 

25 while maintaining a high degree of specificity of degrees of freedom for ambient lighting 
with a low required computational burden. This allows ambient light sources with small 
color gamuts and reduced luminance spaces to emulate video scene light from more 
advanced light sources with relatively large colors gamuts and luminance response curves. 
Possible light sources for ambient lighting can include any number of known lighting 

30 devices, including LEDs (Light Emitting Diodes) and related semiconductor radiators; 
electroluminescent devices including non-semiconductor types; incandescent lamps, 
including modified types using halogens or advanced chemistries; ion discharge lamps, 
including fluorescent and neon lamps; lasers; light sources that are modulated, such as by 
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use of LCDs (liquid crystal displays) or other light modulators; photoluminescent emitters, 
or any number of known controllable light sources, including arrays that functionally 
resemble displays. 

Now referring to FIG. 1, a simple front surface view of a video display D 
5 according to the invention is shown illustratively. Display D can be any of a number of 
known devices which decode video content from a rendered color space, such as an NTSC, 
PAL or SECAM broadcast standard, or an rendered RGB space, such as Adobe RGB. 
Display D comprises color information extraction regions Rl, R2, R3, R4, R5, and R6 
whose borders are arbitrarily pre-defined and which are to be characterized for the piupose 

10 of producing characteristic ambient light A8, such as via back-mounted controllable 

ambient lighting units (not shown) which produce and broadcast ambient light LI, L2, L3, 
L4, L5, and L6 as shown, such as by partial light spillage to a wall (not shown) on which 
display D is mounted. Alternatively, a display frame Df as shown can itself also comprise 
ambient lighting units which display light in a similar manner, including outward toward a 

15 viewer (not shown). If desired^ each color information extraction region Rl - R6 can 

influence ambient light adjacent itself. For example, color information extraction region R4 
can influence ambient light L4 as shown. 

Now referring to FIG. 2, a downward view — part schematic and part cross- 
sectional — is shown of a room or ambient space AO in which ambient light from multiple 

20 ambient light sources is produced using the invention. In ambient space AO is arranged 
. seating and tables 7 as shown which are arrayed to allow viewing of video display D. In 
ambient space AO are also arrayed a plurality of ambient light xmits which are optionally 
controlled using the instant invention, including light speakers 1 - 4 as shown, a sublight SL 
under a sofa or seat as shown, as well as a set of special emulative ambient light units 

25 arrayed about display D, namely center lights that produce ambient light Lx like that shown 
in FIG. 1. Each of these ambient light units can emit ambient light A8, shown as shading 
in the figure. 

In cooperation with the instant invention, one can produce ambient light from 
these ambient light units with colors or chromaticities derived from, but not actually 
30 broadcast by video display D. This allows exploiting characteristics of the human eye and 
visual system. It should be noted that the luminosity function of the human visual system, 
which gives detection sensitivity for various visible wavelengths, changes as a function of 
light levels. 
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For example, scotopic or night vision relying on rods tends to be more sensitive 
to blues and greens. Photopic vision using cones is better suited to detect longer 
wavelength light such as reds and yellowrs. In a darkened home theatre environment, such 
changes in relative luminosity of different colors as a function of light level can be 
5 counteracted somewhat by modulating or changing color delivered to the video user in 
ambient space. This can be done by subtracting light from ambient light units such as light 
speakers 1-4 using a light modulator (not shown) or by use of an added component in the 
light speakers, namely a photoluminescent emitter to further modify light before ambient 
release. The photoluminescent emitter performs a color transformation by absorbing or 

10 undergoing excitation from incoming light from light source and then re-emitting that light 
in higher desired wavelengths. This excitation and re-emission by a photoluminescent 
emitter, such as a fluorescent pigment, can allow rendering of new colors not originally 
present in the original video image or light source, and perhaps also not in the range of 
colors or color gamut inherent to the operation of the display D. This can be helpful for 

1 5 when the desired luminance of ambient light Lx is low, such as during very dark scenes, 
and the desired level of perception is higher than that normally achieved without light 
modification. 

The production of new colors can provide new and interesting visual effects. The 
illustrative example can be the production of orange light, such as what is termed himter's 

20 orange, for which available fluorescent pigments are well known (see ref[2]). The example 
given involves a fluorescent color, as opposed to the general phenomenon of fluorescence 
and related phenomena. Using a fluorescent orange or other fluorescent dye species can be 
particularly useful for low light conditions, where a boost in reds and oranges can 
counteract the decreased sensitivity of scotopic vision for long wavelengths. 

25 Fluorescent dyes that can be used in ambient light imits can include known dyes 

in dye classes such as Perylenes, Naphthalimides, Coumarins, Thioxanthenes, 
Anthraquinones, Thioindigoids, and proprietary dye classes such as those manufactured by 
the Day-Gio Color Corporation, Cleveland, Ohio, USA. Colors available include Apache 
Yellow, Tigris Yellow, Savannah Yellow, Pocono Yellow, Mohawk Yellow, Potomac 

30 Yellow, Marigold Orange, Ottawa Red, Volga Red, Salmon Pink, and Columbia Blue. 

These dye classes can be incorporated into resins, such as PS, PET, and ABS using known 
processes. 

Fluorescent dyes and materials have enhanced visual effects because they can be 
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engineered to be considerably brighter than nonfluorescent materials of the same 
chromaticity. So-called durability problems of traditional organic pigments used to 
generate fluorescent colors have largely been solved in the last two decades, as 
technological advances have resulted in the development of durable fluorescent pigments 
5 that maintain their vivid coloration for 7-1 0 years under exposure to the sun. These 

pigments are therefore almost indestructible in a home theatre environment where UV ray 
entry is minimal. 

Alternatively, fluorescent photopigments can be used, and they work simply by 
absorbing short wavelength light, and re-emitting this light as a longer wavelength such as 

10 red or orange. Technologically advanced inorganic pigments are now readily available that 
undergo excitation using visible light, such as blues and violets, e.g., 400-440 nm light 

Goniophotometric and goniochromatic effects can similarly be deployed to 
produce different light colors, intensity, and character as a function of viewing angles. To 
realize this effect, ambient light units 1-4 and SL and Lx can use known goniophotometric' 

1 5 elements (not shown), alone, or in combination, such as metallic and pearlescent 

transmissive colorants; iridescent materials using well-known diffractive or thin-film 
interference effects, e.g., using fish scale essence; thin flakes of guanine; or 2- 
aminohypoxanthine with preservative. Diffusers using finely ground mica or other 
substances can be used, such as pearlescent materials made from oxide layers, bomite or 

20 peacock ore; metal flakes, glass flakes, or plastic flakes; particulate matter; oil; ground 
glass, and ground plastics. 

Now referring FIG. 3, a system according to the invention to extract color 
information and effect color space transformations to allow driving an ambient light source 
is shown. As a first step, color information is extracted from a video signal AVS using 

25 known techniques. 

Video signal AVS can comprise known digital data frames or packets like those 
used for MPEG encoding, audio PCM encoding, etc. One can use known encoding 
schemes for data packets such as program streams with variable length data packets, or 
transport streams which divide data packets evenly, or other schemes such single program 

30 transport streams. Alternately, the functional steps or blocks given in this disclosure can be 
emulated using computer code and other communications standards, including 
asynchronous protocols. 

As a general example, the video signal AVS as shown can undergo video content 
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analysis CA as shown, using known methods to record and transfer selected content to and 
.from a hard disk HD as shown, possibly using a library of content types or other 
information stored in a memory MEM as shown. This can allow independent, parallel, 
direct, delayed, continuous, periodic, or aperiodic transfer of selected video content. From 
5 this video content one can perform feature extraction FE as shown, such as deriving color 
information. This color information is still encoded in a rendered color space, and is then 
transformed to an unrendered color space, such as CIE XYZ using a RUR Mapping 
Transformation Circuit 10 as shown. RUR herein stands for the desired transformation 
type, namely, rendered-unrendered-rendered, and thus RUR Mapping Transformation 

1 0 Circuit 10 also further transforms the color information to a second rendered color space so 
formed as to allow driving said ambient light source or sources 88 as shown. 

RUR Mapping Transformation Circuit 10 can be functionally contained in a 
computer system which uses software to perform the same functions, but in the case of 
decoding packetized information sent by a data transmission protocol, there could be 

15 memory (not shown) in the circuit 10 which contains, or is updated to contain, information 
that correlates to or provides video rendered color space coefficients and the like. This 
newly created second rendered color space is appropriate and desired to drive ambient light 
source 88 (such as shown in FIGS. 1 and 2), and is fed using known encoding to ambient 
lighting production circuit 18 as shown. Ambient lighting production circuit 18 takes the 

20 second rendered color space information from RUR Mapping Transformation Circuit 10 * 
and then accounts for any input from any user interface and any resultant preferences 
memory (shown together as U2) to develop actual ambient light output control parameters 
(such as applied voltages) after possibly consulting an ambient lighting (second rendered) 
color space lookup table LUX as shown. The ambient light output control parameters 

25 generated by ambient lighting production circuit 18 are fed as shown to lamp interface 
drivers D88 to directly control or feed ambient light source 88 as shown, which can 
comprise individual ambient light units 1 - N, such as previously cited ambient light 
speakers 1 - 4 or ambient center lights Lx as shown in FIGS. 1 and 2. 

To reduce the computational burden, the color information removed from video 

30 signal AVS can be abbreviated or limited. Now referring to FIG.4, an equation for 
calculating average color information from a video extraction region is shown. It is 
contemplated, as mentioned below (see FIG. 14), that the video content in video signal AVS 
will comprise a series of time sequenced video frames, but this is not required. For each 
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video frame or equivalent temporal block, one can extract average or other color 
information from each extraction region (e.g., R4). Each extraction region can be set to 
have a certain size, such as 100 by 376 pixels. Assuming, for example, a frame rate of 25 
frame/sec, the resultant gross data for extraction regions Rl - R6 before extracting an 
5 average (assuming only one byte needed to specify 8 bit color) would be 6 x 100 x 376 x 25 
or 5.64 million bytes/sec for each video RGB tristimulus primary. This data stream is very 
large and would be difficult to handle at RUR Mapping Transformation Circuit 10, so 
extraction of an average color for each extraction region Rl - R6 can be effected during 
Feature Extraction FE. Specifically, as shown one can sum the RGB color channel value 

10 (e.g., Rij) for each pixel in each extraction region of m x n pixels, and divide by the number 
of pixels m X n to airive at an average for each RGB primary, e.g., Ravg for red, as shown. 
Thus repeating this summation for each RGB color chaxmel, the average for each extraction 
region would be a triplet Ravg = |Ravg> Gavg> Bmrgl. The same procedure is repeated for all 
extraction regions Rl - R6 and for each RGB color channel. The number and size of 

1 5 extractive regions can depart from that shown, and be as desired. 

The next step of performing color mapping transformations by RUR Mapping 
. Transformation Circuit 10 can be illustratively shown and expressed using known 

tristimulus primary matrices, such as shown in FIG. 5, where a rendered tristimulus color 
space with vectors R, G, and B is transformed using the tristimulus primary matrix M with : 

20 elements such as Xr,max, Yr,inia, ^r,mm where Xr,niia is tristimulus value of the R primary at 
maximum output. 

The transformation from a rendered color space to unrendered, device- 
independent space can be image and/or device specific — known linearization, pixel 
reconstruction (if necessary), and white point selection steps can be effected, followed by a 

25 matrix conversion. In this case, we simply elect to adopt the rendered video output space as 
a starting point for transformation to an unrendered color space colorimetry. Unrendered 
images need to go through additional transforms to make them viewable or printable, and 
the RUR transformation involves a transform to a second rendered color space. 

As a first possible step, FIGS. 6 and 7 show matrix equations for mapping the 

30 video rendered color space, expressed by primaries R, G, and B and ambient lighting 

rendered color space, expressed by primaries R% G% and B' respectively, into unrendered 
color space X, Y, and Z as shown, where tristimulus primary matrix Mi transforms video 
RGB into unrendered XYZ, and tristimulus primary matrix M2 transforms ambient light 
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source R'G'B* into unrendered XYZ color space as shown. Equating both rendered color 
spaces RGB and R'G'B' as shown in FIG. 8 allows matrix transformations of primaries 
RGB and R'G*B' of the rendered (video) color space and second rendered (ambient) color 
space to said unrendered color space (the RUR Mapping Transformation) using the first and 
5 second tristimulus primary matrices (Mi, M2);and deriving a transformation of color 

information into the second rendered color space (R'G*B') by matrix multiplication of the 
RGB primaries of the rendered video color space, the first tristimulus matrix Mi, and the 
inverse of the second tristimulus matrix (Mz)"^ While the tristimulus primary matrix for 
known display devices is readily available, that for the ambient light source can be 

10 determined using a known white point method by those of ordinary skill in the art. 

Now referring to FIGS. 9-11, prior art derivation of a generalized tristimulus 
primary matrix M using a white point method Is shown. In FIG. 9, quantities like SrXr 
represents the tristimulus value of each (ambient light source) primary at maximum output, 
with Sr representing a white point amplitude, and Xr representing the chromaticities of 

1 5 primary light produced by the (ambient) light source. Using the white point method, the 
' matrix equation equating Sr with a vector of the white point reference values using a known 
inverse of a light source chromaticity matrix as shown. FIG. 11 is an algebraic 
manipulation to remind that the white point reference values such as are a product of the 
white point amplitudes or luminances and the light source chromaticities. Throughout, the 

20 tristimulus value X is set equal to chromaticity a^; tristimulus value Y is set equal to 

chromaticity y; and tristimulus value Z is defined to be set equal to l'(x + y). As is known, 
the color primaries and reference white color components for the second rendered ambient 
light source color space can be acquired using known techniques, such as by using a color 
spectrometer, 

25 Similar quantities for the first rendered video color space can be found. For 

example, it is known that contemporary studio monitors have slightly different standards in 
North America, Europe, and Japan. However, international agreement has been obtained on 
primaries for high-defmition television (HDTV), and these primaries are closely 
representative of contemporary monitors in studio video, computing, and computer 

30 graphics. The standard is formally denoted ITU-R Recommendation BT.709, which 

contains the required parameters, where the relevant tristimulus primary matrix (M) for 
RGB is: 

0.640 0.300 0. 1 50 Matrix M for ITU-R BT.709 
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0.330 0.600 0.060 
0.030 0.100 0.790 

and the white point values are known as well. 

Now referring to FIG. 12, a system similar to that shown in FIG. 3 is shown, 
5 additionally comprising a gamma correction step 55 after feature extraction step FE as 
shown for ambient broadcast. Alternatively, gamma correction step 55 can be performed 
between the steps performed by RTJR Mapping Transformation Circuit 10 and Ambient 
Lighting Production Circuit 18. Optimum gamma values for LED ambient light sources has 
been found to be 1.8, so a negative gamma correction to counteract a typical video color 
1 0 space gamma of 2.5 can be effected with the exact gamma value found using known 
mathematics. 

Generally, RUR Mapping Transformation Circuit 10, which can be a functional 
block effected via any suitable known software platform, performs a general RUR 
transformation as shown in FIG. 13, where a schematic as shown takes video signal AVS 

1 5 comprising a Rendered Color Space such as Video RGB, and transforms it to an 
unrendered color space such as CIE XYZ; then to a Second Rendered Color Space 
(Ambient Light Source. RGB). After this RUR transformation, ambient light sources 88 can 
be driven, aside from signal processing, as shown. 

FIG. 14 shows the top of FIGS. 3 and 12, where an alternative extraction step is 

20 shown whereby a frame decoder FD is used. Regional Information is extracted at step 33 as 
shown, and output calculations 00, such as the averaging of FIG. 4, is performed prior to 
data transfer to Ambient Lighting and Production Circuit 18 previously shown. 

As shown in FIG. 15, general process steps include Acquiring an video signal 
AVS; Extracting Regional (Color) Information; an RUR Mapping Transformation; optional 

25 Gamma Correction; and Driving Light Sources. Generally, FIG. 16 shows process steps for 
acquiring transformation matrix coefficients for an ambient light source used by the 
invention, where the steps include, as shown. Driving the ambient light unit(s); and 
Checking Output Linearity as known in the art. If the ambient light source primaries are 
stable, (shown on left fork. Stable Primaries), one can Acquire Transformation Matrix 

30 Coefficients Using a Color Spectrometer; whereas if the ambient light source primaries are 
not stable, (shown on right fork. Unstable Primaries), one can reset the previously given 
gamma correction (shown, Reset Gamma Curve). 

In general, it is desirable, but not necessary to extract color information from 
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every pixel in extraction regions such as R4, and instead polling of selected pixels can allow 
a faster estimation of average color, or a faster creation of a extraction region color 
characterization, to take place. FIG. 17 shows process, steps for estimated video extraction 
and ambient light reproduction using the invention, where steps include [1] Prepare 
5 Colorimetric Estimate of Video Reproduction (From Rendered Color Space, e.g.. 
Video RGB); [2] Transform to Unrendered Color Space; and [3] Transform 
Colorimetric Estimate for Ambient Reproduction (Second Rendered Color Space, e.g., 
LED RGB). 

Generally, ambient light source 88 can embody various diffuser effects to 

1 0 produce light mixing, as well as translucence or other phenomena, such as by use of lamp 
structures having a frosted or glazed surface; ribbed glass or plastic; or apertured structures, 
such as by using metal structures surrounding an individual light source. To provide 
interesting effects, any number of known diffusing or scattering materials or phenomena 
can be used, including that obtain by exploiting scattering from small suspended particles; 

15 clouded plastics or resins, preparations using colloids, emulsions, or globules 1-5 :m or less, 
such as less than 1 :m, including long-life organic mixtures; gels; and sols, the production 
and fabrication of which is known by those skilled in the art. Scattering phenomena can be 
engineered to include Rayleigh scattering for visible wavelengths, such as for blue 
production for blue enhancement of ambient light. The colors produced can be defined 

20 regionally, such as an overall bluish tint in certain areas or regional tints, such as a blue 
light-producing top section (ambient light LI or L2). 

Ambient lamps can also be fitted with a goniophotometric element, such as a 
cylindrical prism or lens which can be formed within, integral to, or inserted within a lamp • 
structure. This can allow special effects where the character of the light produced changes 

25 as a function of the position of the viewer. Other optical shapes and forms can be used, 
including rectangular, triangular or irregulariy-^shaped prisms or shapes, and they can be 
placed upon or integral to an ambient light unit or units. The result is that rather than 
yielding an isotropic output, the effect gained can be infinitely varied, e.g., bands of 
interesting light cast on surrounding walls, objects, and surfaces placed about an ambient 

30 light source, making a sort of light show in a darkened room as the scene elements, color, 
and intensity change on a video display unit. The effect can be a theatrical ambient lighting 
element which changes light character very sensitively as a fimction of viewer position - 
such as viewing bluish sparkles, then red light - when one is getting up from a chair or 
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shifting viewing position when watching a home theatre. The number and type of 
goniophotometric elements that can be used is nearly unlimited, including pieces of plastic, 
glass, and the optical effects produced from scoring and mildly destructive fabrication 
techniques. Ambient lamps can be made to be unique, and even interchangeable, for 
5 different theatrical effects. And these effects can be modulatable, such as by changing the 
amount of light allowed to pass through a goniophotometric element, or by illuminating 
different portions (e.g., using sublamps or groups of LEDs) of an ambient light unit. 

In this way, ambient light produced at L3 to emulate extraction region R3 as 
shown in FIG. 1 can have a chromaticity that provides a perceptual extension of a 
1 0 phenomenon in that region, such as the moving fish as shown. This can multiply the visual 
experience and provide hues which are ^propriate and not garish or unduly mismatched. 

Video signal AVS can of course be a digital datastream and contain 
synchronization bits and concatenation bits; parity bits; error codes; interleaving; special 
modulation; burst headers, and desired metadata such as a description of the ambient 
1 5 lighting effect (e.g., "lightning storm"; "sunrise"; etc.) and those skilled in the art will 

realize that functional steps given here are merely illustrative and do not include, for clarity, 
conventional steps or data. 

The User Interface & Preferences Memory as shown in FIGS. 3 and 12 can be 
used to change preferences regarding the system behavior, such as changing the degree of 
20 color fidelity to the video content of video display D desired; changing flamboyance, - 
including the extent to which any fluorescent colors or out-of-gamut colors are broadcast 
into ambient space, or how quickly or greatly responsive to changes in video content the 
ambient light is, such as by exaggerating the intensity or other quality of changes in the 
light script command content. This can include advanced content analysis which can make 
25 subdued tones for movies or content of certain character. Video content containing many 
dark scenes in content can influence behavior of the ambient light source 88, causing a 
dimming of broadcast ambient light, while flamboyant or bright tones can be used for 
certain other content, like lots of flesh tone or bright scenes (a sunny beach, a tiger on 
savannah, etc.). 

30 The description is given here to enable those of ordinary skill in the art to 

practice the invention. Many configurations are possible using the instant teachings, and 
the configurations and arrangements given here are only illustrative. In practice, the 
methods taught and claimed might appear as part of a larger system, such as an 
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entertainment center or home theatre center. 

It is well known that for the functions and calculations illustratively taught here 
can be functionally reproduced or emulated using software or machine code, and those of 
ordinary skill in the art will be able to use these teachings regardless of the way that the 
5 encoding and decoding taught here is managed. 

Those with ordinary skill in the art will, based on these teachings, be able to 
modify the apparatus and methods taught and claimed here and thus, for example, re- 
arrange steps or data structures to suit specific applications, and creating systems that may 
bear little resemblance to those chosen for illustrative purposes here. 
10 The invention as disclosed using the above examples may be practiced using 

only some of the features mentioned above. Also, nothing as taught and claimed here shall 
preclude addition of other structures or functional elements. 

Obviously, many modifications and variations of the present invention are 
possible in light of the above teaching. It is therefore to be imderstood that, within the 
15 scope of the appended claims, the invention may be practiced otherwise than as specifically 
described or suggested here. 



